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ABSTRACT 

The purpose of this study is to assess the impact of a newly developed modules, Interactive Web-Based Visualization 
Tools for Gluing Undergraduate Fuel Cell Systems Courses system (IGLU), on learning motivations of engineering 
students using two samples (ni=144 and n 2 = 135) from senior engineering classes. The multivariate analysis results 
revealed that the participants had a significant increase in their learning motivation after the treatment with the 
IGLU modules. This result was cross-validated with the two samples, in which the motivation mean posttest scores 
are significantly higher than the mean pretest scores, systematically (Sample 1: the mean score is increased by 2.09 
[.32, 3.87] points, p = .021; Sample 2: the mean score is increased by 1.38 [.14, 2.61] points, p = .029). With the 
use of instructional technology> prevailing in current university courses, the education initiative of the IGLU system 
and the assessment of its impact on student learning motivation provide us information to improve the modules to 
serve a more diverse student body. It will greatly help the development of engineering educational curriculum. With 
regards to the statistical inference, it is desirable to conduct further studies with a quasi-experiment control group 
design to assess the program effect focusing on student learning and its associations with student learning 
motivations and learning styles. 
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INTRODUCTION 

quasi-experimental research study was conducted to assess the effectiveness of a newly developed 
web-based modules on learning motivations and outcomes of undergraduate engineering students. 
The participants were from the college of engineering at a large, urban research university in the 
southeastern United States. To meet urgent industry needs for engineers who are educated in renewables energy 
technologies with system level thinking, the researchers of this project created a program, named Interactive Web- 
Based Visualization Tools for Gluing Undergraduate Fuel Cell Systems Courses (IGLU). The interactive and 
interconnected visualization modules in the software were created to enhance undergraduate students’ motivations 
toward learning engineering courses. One of the project objectives is to assess the impact of IGLU program on 
learning motivations of engineering students. 

Following the introduction, we have included sections on the background of study, supporting literature on 
instructional technology as the means to promote student learning motivation, summary of the development 
of the overall study including the research design, instrumentation, data analysis results, discussion, and 
conclusions of the study. 
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BACKGROUND 

To answer the call of the National Science Foundation (NSF) for transforming undergraduate education in Science, 
Technology, Engineering, and Mathematics (STEM) program to improve the quality of STEM education for all 
undergraduate students, the research team created the IGLU program under the NSF grant support that started in 
2013. The focus of the IGLU program was on transforming undergraduate engineering education through bringing 
about potential widespread adoption of classroom practices using instructional technology. 

The program supports efforts to create, adapt, and disseminate new learning materials and teaching strategies to 
reflect the advances both in STEM disciplines, and in what is known about teaching and learning practices. The 
project involved implementation of the educational innovation, and evaluation of the innovation through quasi- 
experimental design to study student motivation and learning. 

Specifically, in response to promoting undergraduate education in engineering courses, the topic on Fuel Cells is 
selected as the ideal course for designing the IGLU project. As wc know, due to limited petroleum resources and 
environmental considerations, research investigation and development of different alternative energy sources have 
been a hot topic in recent years. The fuel cell is an electrochemical conversion device that could use hydrogen or 
other hydrocarbon fuels to produce electricity, with water as the byproduct (if hydrogen is used as the fuel), which 
has been increasingly used in automobiles, hospitals, hotels, power plants, aircrafts, and shuttles, just to name a 
few. However, the progress in fuel cell systems as well as their efficiency and robustness depend not only on the 
current quality of science and engineering research and development, but also on the quality and preparedness of 
our next generation students, who may be involved in fuel cells research and development in the future. Currently, 
there are two major challenges facing the educators in Engineering: the decreasing number of student enrollment 
and graduation rates (Yarmolenko, et ah, 2009). 

Figure 1 . A few screenshots of animations and movies in the educational software. (A) “How does a fuel cell work?” part of the 
introduction module; (B) screenshot of the “Electro-osmotic drag” animation; (C) design of tubular fuel cell stacks; (D) hybrid 
system of batteries and fuel cells in unmanned aerial vehicles; (E) equation used in reaction kinetics science of fuel cells; and (F) 
picture and video showing the fuel cell production process. 
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The trend of the student enrollment and graduation rate is not only related to the level of complexity associated 
with science and engineering subjects, but also the lack of proper tools to improve students’ motivations to 
learn and make students aware about the possible direction of their future careers. Therefore, there is an 
urgent need for engineering educators to develop innovative teaching approaches to better suit the learning styles 
of the new generation of technologically savvy engineering students, to arouse their interest in the engineering 
courses. 


At the same time, for those students who are already in engineering schools across the US, there is a lack of 
understanding of the “bigger picture” for system concepts. NSB (2015) stated, “U.S. engineers need to excel at 
high-level design, systems integration, innovation, and leadership” (p. 14). However, engineering courses such as 
fuel cell related courses are sometimes technically too intensive for the undergraduate level students and could be 
too focused on particular aspects of the fuel cell system rather than giving them the broad picture which helps them 
develop system-level thinking. This significantly impairs student learning motivation for engineering courses and 
their retention in the engineering programs. 

The key question that the IGLU addresses is “How to motivate our next generation engineers in learning 
techniques used in fuel cell systems and grow them with system level thinking?” Our answer is: to achieve the goal 
through the interactive and technically interconnected visualization tools augmented with hardware 
demonstrations, which will have sufficient level of technical sophistication for undergraduate students, yet are 
clear and precise enough to keep them interested in the subject and be actively involved in learning. We believe that 
such education is “learning based” instead of “teaching based”, and it will help motivate the students to stay in the 
program and excel. 


Figure 2. From left to right, the figure shows the main interface of the software (A), followed by the components of the Fuel Cell 
System module (B) and finally the Fuel Cell Processing Suh-system (C). 
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Furthermore, recent studies have shown that interactive visualization tools can improve the educational experience 
for engineering and computer science students (e.g., Messner & Horman, 2003; Mohler, 2011; Nguyen & Khoo, 
2009). Therefore, web-based interactive instruments are expected to play a significant role in attracting students to 
leam and to complete their courses successfully. However, to date, there are very limited resources available 
specifically for engineering education, such as tools that are suitable for fuel cell related undergraduate courses. 
Therefore, IGLU specifically aims at building an essential link that is currently missing: the link between 
understanding fuel cell science and technology to understanding of the overall fuel cell system. 

Taking into consideration all these points in the IGLU project (Aman, Xu, Bai, Orlovskaya, 2015), we focused the 
learning modules to cover all aspects of a fuel cell system and broke it down into five main modules: “introduction”, 
“applications”, “fuel cell system”, “cell level”, and “fuel cell science”. These modules are highly interactive and 
interconnected for improving college engineering students’ understanding of fuel cells knowledge at various levels. 
Based on Felder and Silverman (1988), “Most people of college age and older are visual [learners]” (p. 676), we 
designed the modules with active animations to transform the abstract concepts into a composition of small parts 
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that not only build on top of previous concepts linearly but are also inter-connected in a way to give students the 
freedom the navigate the topics as they see fit. 

We also made the modules in a way that they directly map the student learning outcomes to Bloom’s taxonomy 
(Anderson & Krathwohl, 2001). We used ARCS model (Keller, 1987) to guide us to design the visualized and 
interactive modules for increasing student motivation in learning fuel cell knowledge through the interactive 
diagrams such as presented in Figure 1 depicting the sub-system and system-level components of the fuel cell 
system (Bai, Xu, Aman & Orlovskaya, 2014). As mentioned earlier, the entire IGLU software is comprised of five 
main modules, each module addressing the concepts related to fuel cell systems at different levels. Each module in 
turn is composed of various sub-modules. For example, the Introduction module includes topics concerning the 
definition of a fuel cell, working principles, types and comparisons of fuel cell systems to other energy conversion 
technologies; that gives the student a quick overview of what they need to know about fuel cells. Applications 
module goes into details about how and where fuel cell systems are currently being used and the advantages of 
doing so. The third module, the Fuel Cell Systems module, talks about the different sub-systems that make up a 
complete fuel cell system. Figure 2 shows the one of the components of the Fuel Cell System module along with the 
main software interface. The Cell Level module discusses the components of a single cell, how they are produced 
and other key concepts. Finally the Fuel Cell Science module deals with all the fundamental science behind the 
working of a fuel cell such as thermodynamics, reaction kinetics, charge and mass transport in the cell. The user 
interface is simple in design and is made of clickable links connected to different parts of the software. Links 
throughout the software enable the user to jump from one topic to another based on their interest or learning pattern. 
The software can be accessed via any web browser through a specific server or a website. In addition to animations, 
there are also videos and some of the animations are accompanied by audio narratives. The dynamic components of 
the software are designed to keep the students engaged and motivated to learn, at the same time keeping each 
animation/video short enough will demand less attention span from the student which could help in higher retention. 
The intention is that the software is used as a supplementary tool to enhance student learning and as such could be 
used in the classroom to further illustrate concepts or as a bonus outside-the-classroom learning tool or even the 
basis of a flipped classroom method. 

The purpose of this study is to assess the effects of a newly developed web-based IGLU system on learning 
motivations of engineering students. With the online learning prevailing in current university courses, we expect 
that the education initiative of the IGLU system and assessment of its impact on student learning motivation will 
provide information for improving the modules to serve a more diverse student body. It will greatly help 
curriculum development for engineering education. 

FRAMEWORK AND SUPPORTING LITERATURE 
Motivation and Learning Interest 

How to motivate student to learn has long been recognized an important question for educators and researchers. In 
the vast literature of motivation theory, it is well known that intrinsic and extrinsic motivation play an essential role 
in developmental and educational practices (Rayan & Deci, 2000). Intrinsic motivation refers to the motivation 
derived from personal interest which generates from inside, while extrinsic motivation refers to doing things for 
specific outcomes which arises from outside of an individual (Deci & Ryan, 1975). Students’ intrinsic motivation 
usually leads them to engage in learning in different ways and persists in solving problems, while students with 
extrinsic motivation may attempt to finish the tasks as a goal rather than enjoying the process of learning; as a result, 
students may begin a task actively but give-up when they come across difficulties (Deci & Ryan, 1975). However, 
the important aspect of the relationship between the two distinguished types of motivations is that extrinsic 
motivation can reinforce intrinsically motivated behaviors, which is mediated through stimulating personal interests 
(Deci, Koestner, & Ryan, 1999). The mediation approach has promoted the educational practices for educators to 
explore strategies to motivate students to learn through triggering their learning interests. 

Early from 1900s’, Dewey's (1913) pioneering work on the role of interest in learning orientated educators and 
researchers to focus their efforts in educational practices and research in this area. Even though there exists a large 
amount of literature regarding teaching effort and student interest, Dewey’s (1913) argument had been the explicit 
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and leading theory orientating researchers and educators in this field for about 50 years until 1980s. The major 
approach of Dewey (1913) is that if teaching can secure interest in their given set of factors, such as format and 
materials, the students are sure to direct their motivations towards mastering them, while if the teaching factors are 
not guaranteed interest, educators may not be able to ensure students willingly to learn in any given cases, or in 
other words, students will not learn well when they go at it without putting their hearts in it. 

Following Dewey’s argument on the leading role of student interest in motivating them in learning, researchers 
started to further detail the relationships between interest and learning motivations since 1980s. Kintsch’s (1980) 
study was one of the most influential approaches to address the association between interest and learning. Kintsch 
(1980) theorized the concept of situational interest and its relationship with other factors related to learning. 
Situational interest is defined as provisional interest that arises spontaneously due to environmental factors such as 
task directions or an appealing text (Schraw, Flowerday, & Lehman, 2001). According to Kintsch (1980), there are 
two types of situational interest as emotional and cognitive interest. Emotional interest was likely to be aroused by 
information with strong affective response such as delightfulness or irritation, while cognitive interest arises when 
learners get involved in event or activities. 

Subramaniam (2010) pointed out that situational interest played the key role as a motivator in enhancing student 
engagement in learning process. Educational studies in this area focus more on the effect of situational interest on 
motivating student learning because it is the factor that can be managed and manipulated by educators. Researchers 
(e.g., Hidi & Baird, 1986; Mitchell, 1993) emphasized that situational interest appeared to be especially important in 
catching students’ attention, and hereafter to initiate personal interest in holding their motivation in learning. Hidi 
(1990) proposed that situational interest increased learning when learning tasks or instructional materials were 
innovative. Other researchers (e.g., Schraw & Dennison, 1994; Shirey, 1992) also empathize that student learning 
interest is likely motivated by information pertinent to a task or learning goal. 

With the theoretical guidance of learning interest and motivation, empirical studies have been flourishing in the past 
decades. As presented in Potvin and Hasni (2014), there were 228 studies at k-12 levels since 2000, and in another 
search by the authors of the current study in April 2016, there were 1362 research articles related to the learning 
interest and motivation topics, and 353 of them are at college level studies during 2000-2016 indexed in the ERIC 
database. Among them, there were 295 articles related to the use of technology to enhance student learning interest 
and motivate them to leam better. The large amount of studies on student learning interest and motivation related to 
technology use in the most recent years reflect the current research focuses on student learning and the shift of the 
educational instructions to e-learning environments to promote student learning. 

Learning Motivation and Instructional Technology 

In the e-age, student interest is more diverse than ever before. To motivate student interest in learning academic 
courses has become more difficult. In the past decade, the problem of decreasing of students majoring in science and 
technology in higher education has become even more acute (Osborne, Simon, & Collins, 2003; Potvin & Hasni, 
2014). In the current situation, the question of how to improve student learning motivation through initiating their 
learning interest is imperative. The motivation and interest theory as discussed above has provided guidance for us 
to find answers to the above question. According to the literature, student learning interest can be motivated when 
the instructional materials (1) match their learning styles (Felder & Silverman, 1988), (2) leverage the difficulty 
levels (Cordova & Lepper,1996), (3) have feasible missions or tasks (Flowerday & Schraw, 2000), and (4) arouse 
their curiosity (Kohn, 1993; Lowenstein, 1994). Instructional technology brings more opportunities for teachers to 
manage student situational interest through incorporating the above factors influencing student interest in course 
learning. 

First, teaching based on the strengths of youth, educators need to understand their students’ world and the way they 
learn. In the e-age, one way to motivate student learning interest is to develop instructional strategies that seamlessly 
encompass digital contexts in students’ life to accommodate their learning style (Hull & Shultz, 2002). Moreover, in 
describing the learning styles, Felder and Silverman (1988) states that most students follow the visual, sensing, 
inductive, and active dimensions. With the advance of computer technology and rich instructional technology 
available to course design, it is possible to make teaching and learning more favorable to match students’ learning 
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styles. It is clear that digital tools can help with visual learners through presenting live pictures or illustrating the 
interactive procedures for some scientific process which may not be available to be seen in either a classical 
classroom or lab. 

Second, technology can leverage the learning difficulties on learning some abstract concepts (Mishra & Koehler, 
2006). The use of technology can make students learn interactively and easily through digital demonstrations or 
interactive visualization tools. For example we can design learning modules to make student easily navigate through 
interactive visualization tool, find the learning materials that they are interested in at that moment, input their own 
answers, provide their suggestions/comments, learn an interactive forum. Furthermore, hardware demonstrations 
and experiments can be incorporated as movies in the interactive visualization tools for online learning to help 
student learn difficult contents (Heafner, 2004). 

Third, technology can segment an ambitious learning goal into interconnected feasible specific objectives to 
maintain student learning interest (Raman, Ryan, & Olfman, 2005). While the technology can be used to fit well 
with the learning style of students (Dede, 2004), it can help to make learning objectives achievable, such as through 
interactive procedures with interconnected single missions, and therefore to improve student learning efficiency. 
Technology can also give immediate and interactive feedbacks to students’ online activities; hence, it could 
increase students’ confidence level (Mun & Hwang, 2003) for them to maintain their motivation in achieving their 
learning goal. 

Last but not the least, technology can arouse student curiosity in learning. The use of technology can provide a non¬ 
threatening and fun learning environment for all students and incorporate self-regulated learning (SRL) strategies 
(Zimmerman & Schunk, 1989) that usually prompt learners’ curiosity and interest to think and reflect critically 
about how they learn. 

With regards to the opportunities using technology to help with students learning through enhancing their learning 
motivations, we would like to explore the answers to the following research questions in this study: 

• Are there any significant differences in learning motivation changes of engineering students over the 
IGLU intervention period controlling for their demographic variables (e.g. gender and ethnicity)? 

• What are the significant changes in student course learning motivation after the students attending the 
IGLU intervention if any? 

Method 

The quasi-experimental pre and posttest research design is used to assess the effect of interactive learning module 
with technology assistance on engineering student learning motivation. The repeated measure multivariate analysis 
of covariance (MANCOVA) are used to answer our research questions. The strength of the current study is that we 
were able to use two empirical samples to cross-validate the statistical analysis results. 

Participants 

Two random clustered samples of engineering college students from a large university in the southeastern United 
States are used for the current study with sample size of 144 {Data 1 ) and 135 {Data 2) from 2014 fall semester and 
2015 spring semester respectively. The students were enrolled in senior engineering classes. There are some missing 
data from Dataset 1. We examined the missing values and found that there was no systematic missing at pretest or 
protest; therefore, we deleted the 22 cases with random missing values and kept 122 cases for the final analysis. 
After deleting the missing data, Dataset 1 consists 18 females (104 male), 117 students with age from 20-29 and 5 
equal or above 30, 7 students with learning disabilities, and 120 from senior classes, 79 Caucasian, 25 Hispanic, and 
20 others. Dataset 2 consists 36 females (99 male), 125 students with age from 20-29 and 10 above 30, 7 students 
with learning disabilities, and 132 from senior class, 80 Caucasian, 33 Hispanic, and 22 others. 
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Instrumentation 

The Course Motivation Scale (CMS, Kebritchi, Hirumi, & Bai, 2010) was used to collect student learning 
motivation data, and the researcher-created Fuel Cell Knowledge Test was used to measure student learning 
outcomes before and after the IGLU intervention period. 

Student Learning Motivation Survey 

The Course Motivation Scale (CMS, Kebritchi, Hirumi, & Bai, 2010) was created based on Keller’s ARCS Model 
(1987). The questionnaire is a 5-point Likert scale (from Not true =1 to Very true =5) with 20 items measuring 
student course motivation on four dimensions, including attention, relevance, confidence and satisfaction based on 
Kebritchi, et al. (2010). Attention measures students’ interest arising and preserving during educational activities, 
and sample questions include, “There is something interesting about this course that will capture my attention,” and 
“I believe that I will enjoy this course so much that I would like to know more about this topic.” Relevance denotes 
whether a student sees the activity pertinent to personal interest, and a sample questions is, “The course is relevant to 
my interests.” Confidence represents whether a student presumes a feasible goal, and sample questions include, “1 
believe that I will be confident in my ability to successfully complete all class assignments and requirements after 
working on this course for a while.” Satisfaction refers to the value that the students see from the course work, and 
sample questions contain “I believe that completing this course will give me a feeling of satisfaction,” and “It is 
apparent to me how people use the information in this course.” 

The content validity of the Motivation survey was examined and confirmed by the experts well understand Keller's 
ARCS model. Exploratory factor analysis using the pretest data from Dataset 1 for this study found clear four-factor 
structures with 62.88% of the total variance explained which confirmed the construct validity. The Cronbach's alpha 
reliability of ARCS measured by the current data was 0.92 for Data 1 with a sample of 122, and .91 for Data 2 with 
a sample of 135. 

Fuel Cell Knowledge Test 

The Fuel Cell Knowledge Test consisted 10 multiple choice questions with the total score of 30. The content validity 
was confirmed by two experts in the Fuel Cell and related engineering subjects. The instrument was also reviewed 
by an expert in measurement and evaluation in education to validate the quality of measure. 

Data Collection Procedure 

The Institutional Review Board (IRB) approval at the authors’ university was obtained before collecting data. The 
IRB procedures were strictly followed during data collections. All participants were secure confidentiality but not 
anonymity because their pretest and posttest data needed to be matched. The participation of the project was on the 
volunteering bases. 

For the first round of data collection, paper-based surveys were given to the students in a normal classroom setting. 
Before the class instruction, the researchers distributed the test papers and questionnaires to the students in one large 
classroom along with the consent form approved by IRB as the cover page of the survey package. One of the 
researchers read the consent form to the students and made students aware that they were invited to participate in the 
research on a volunteer basis. Students were asked to complete the Student Learning Motivation survey and the Fuel 
Cell Knowledge Test as the pretests before class. Then one of the researchers taught the entire class with the 
software modules as the intervention for 25 minutes, and the whole class duration was about 50 minutes. After the 
intervention session, students were asked to complete the Student Learning Motivation survey and the Fuel Cell 
Knowledge Test again as the post-tests. Students’ demographic data, such as gender and ethnicity, were also 
collected. 

For the second round of data collection, we used the online survey. The consent form was also distributed online to 
the students to ensure students were invited to voluntarily participate in the research. The software was used in a 50- 
minute class. The pretest were delivered to students one day before the class meeting time. In class before the 
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instruction the researchers gave 15 minutes for those who had not taken the pretests to complete the pretests online 
using personal laptop computers, iPads or smart phones. The posttests were available 24 hours for students to 
complete online right after the treatment. 


DATA ANALYSIS RESULTS 

Before conducting the multivariate data analysis, both data sets were examined for assumptions. The dependent 
variables, pre and posttest scores of student motivation and knowledge tests were all normally distributed for both 
datasets for this study. Homoscedasticity was also tested, and the equal variance and covariance among the 
dependent variables were assumed for both data sets with Box’s M = 44.51 (iq 30 , 1350 ) = 1.20, p = .21) for Data 1 and 
Box’s M=66A1 (7^(50,2844) = 1.10 ,p = .29) for Data 2 (See Table 1). 

The correlations among the dependent variables were examined. Pretest and posttest motivation scores were highly 
correlated with r = .737 (p < .001) for Data 1 and r = .343 {p < .001) for Data 2. Pretest and posttest knowledge 
scores were also significantly correlated with r = .351 (p < .001) for Data 1 and with r = .852 (p < .001) for Data 2. 
(See Table 2). With repeated measures, multicollinearity is not a concern for testing within subject effect. 


Table 1. Box's Test of Equality of Covariance Matrices 3 



Data 1 

Data 2 

Box's M 

44.509 

66.174 

F 

1.197 

1.102 

dfl 

30 

50 

df2 

1350.506 

2844.004 

P 

.214 

.290 


Note. Tests the null hypothesis that the observed covariance matrices of the dependent variables are equal across groups, a. Design: Intercept + 
Gender + Ethnicity + Gender * Ethnicity; 

Within Subjects Design: Time 


Table 2. Pearson correlations for student motivation scores and knowledge test scores 


Data Source 


CMS_pre_T 
( p-value ) 

CMSpostT 
( p-value ) 

QpreT 
( p-value ) 

Q_post_T 
( p-value ) 

Dataset 1 

CMS_pre_T 

1 

.737" 

mi 

.242" 


(<001) 

(.771) 

(.007) 

CMS_post_T 


1 

.041 

.369" 



(.654) 

(<001) 

(N = 122) 

0 pre T 



1 

.351" 






(<.001) 


O post T 




1 








CMS pre T 

1 

.852" 

.064 

.067 




(<.001) 

(.462) 

(.443) 


CMS post T 


1 

.206* 

.110 

Dataset 2 




(.016) 

(.206) 

(N=135) 

0 pre T 



1 

.343" 






(<001) 


0 post T 




1 








Note. CMS = Motivation test; Q = Knowledge test. 


Repeated measure MANCOVA was conducted first to analyze data to test differences in their motivation scores and 
knowledge test scores between pre- test and post-test while controlling for the covariates, gender and ethnicity. The 
first model showed that there were no gender effect (p = .26 for Data 1 and p = .32 for Data 2) and ethnicity effect 
( p = .96 for Data 1 and p = .50 for Data 2) with all ji-values large than .05; therefore, we modified the model by 
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removing the gender and ethnicity from the model and ran repeated measure MANOVA to only examine the within- 
subject effect for motivation and knowledge test scores presented in Table 3 for both Data 1 and Data 2. The 
descriptive statistics for both data sets are shown in Table 4. 


Table 3. Within-Subjects Factors 


Measure 

Time 

Dependent Variable 

.... .. 

1 

CMS_pre T 


2 

CMS post T 

Knowledge 

1 

Qjre T 

2 

Q post T 


Note. CMS = motivation test; Q = knowledge test 


Table 4. Descriptive Statistics 


Data Source 


Mean 

SD 

N a 


Motivationjre 

67.01 

13.006 

122 


Motivationjost 

69.10 

14.128 

122 


Knowledge pre 

8.221 

4.2456 

122 


Knowledge post 

22.689 

4.3467 

122 


Motivationjre 

70.5407 

13.150 

135 

Data 2 

Motivation_post 

71.9185 

13.511 

135 

Knowledge pre 

11.8000 

5.068 

135 


Knowledge post 

21.578 

4.855 

135 


Note. 1 listwise deletion 


Multivariate analysis results for both Data 1 and Data 2 revealed the significant effect of the combined variable on 
the effect of pre and posttest with Wilks' 1 = .101 (Fp, 120 ) = 533.33, p < .001, if = .899) for Data 1, and Wilks' 1 
=.237 (Fp_ 133 ), p < . 001, tf= .765) for Data 2 (See Table 5). The results indicated that there were statistically 
significant differences between pre and posttest scores on either motivation scores or knowledge tests or both of the 
dependent variables. 


Table 5. Within sub jects effect of Multivariate test results a,b 


Data source 

Wilks' X 

F 

Hypothesis df 

Error df 

P 

>1 2 

Data 1 

.101 

533.330° 

2 

120 

<.001 

.899 

Data 2 

.237 

213.631° 

2 

133 

<.001 

.763 


Note. a. Design: Intercept; Within Subjects Design: time; b. Tests are based on averaged variables; c. Exact statistic 


Table 6, Univariate Tests of Within-Subject Effects 


Data 

Source 

Source 

Measure 

time 

Type III 

Sum of 
Squares 

df 

Mean 

Square 

F 

P 


time 

Motivation 

Linear 

266.496 

1 

266.496 

5.436 

.021 

Data 1 


Knowledge 

Linear 

12767.316 

1 

12767.316 

1066.010 

<.001 

En'or(time) 

motivation 

Linear 

5932.004 

121 

49.025 





Knowledge 

Linear 

1449.184 

121 

11.977 




time 

Motivation 

Linear 

128.133 

1 

128.133 

4.864 

.029 

Data 2 


Knowledge 

Linear 

6453.333 

1 

6453.333 

398.746 

<.001 

Error) time) 

motivation 

Linear 

3529.867 

134 

26.342 





Knowledge 

Linear 

2168.667 

134 

16.184 
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With regards to the multivariate results, we further conducted univariate tests to examine the specific pre and 
posttest differences on each dependent variable. As we can see from Table 6, univariate test of within-subject effects 
on both Data 1 and Data 2 revealed that there were significant pretest and posttest differences for both motivation 
and knowledge test scores. The changes over time of motivation scores were significant at a = .05 level with F { i, 12 i) 
= 5.436 (p = .021) for Data 1 and Fq, i 34) = 4.864 (p = . 029) for Data 2. The changes of knowledge tests were also 
significant with F (l l2 i)= 1066.01 (p < .001) for Data 1 andTq^ i 34 ) = 398.746 (p < .001) for Data 2. (See Table 6) 

Considering the analysis results from univariate tests, we further examine the specific mean differences of post-test 
of motivation scores for both Data 1 and Data 2 on both dependent variables. From Table 7 we can see that the 
mean of motivation posttest (M = 69.10; SD = 14.13) (see Table 4) was at 2.09 [.32, 3.87] points significantly higher 
(See Table 7) than the mean of pre-test score (M = 67.01, SD = 13.01) (see Table 4) in Data 1. The mean of 
motivation posttest (M = 71.92; SD = 13.51) (see Table 4) was at 1.38 [.14, 2.61] points significantly higher (See 
Table 7) than the mean of pre-test score (M = 70.54, SD = 13.51) (see Table 4) in Data 2. 


Table 7. Pairwise Comparisons for the motivation and knowledge test score changes from time 1 to 2 


Data 

source 

Measure 

(I) 

time 

(J) 

time 

Mean 

Difference 

(I-J) 

Std. 

Error 


95% Confidence Interval for 
Difference b 

Lower Bound 

Upper Bound 

Data 1 

Motivation 

1 

2 

-2.090* 

.896 

.021 

-3.865 

-.315 



2 

1 

2.090* 

.896 

.021 

.315 

3.865 


Knowledge 

1 

2 

-14.467* 

.443 

<.001 

-15.344 

-13.590 



2 

1 

14.467* 

.443 

<.001 

13.590 

15.344 

Data 2 

Motivation 

1 

2 

-1.378* 

.625 

.029 

-2.613 

-.142 



2 

1 

1.378* 

.625 

.029 

.142 

2.613 


Knowledge 

1 

2 

-9.778* 

.490 

<.001 

-10.746 

-8.809 



2 

1 

9.778* 

.490 

<.001 

8.809 

10.746 


Note. Based on estimated marginal means*. The mean difference is significant at the .05 level, b. Adjustment for multiple comparisons: Least 
Significant Difference (equivalent to no adjustments). 


We also further examine the specific mean differences of post-test of knowledge scores for both Data 1 and Data 2. 
From Table 7 we can see that the mean of the knowledge posttest (M = 22.69; SD = 4.35) (see Table 4) was at 14.47 
[13.59, 15.34] points significantly higher (See Table 7) than the mean of pre-test score (M= 8.22, SD = 4.25) (see 
Table 4) in Data 1. The mean of knowledge posttest (M = 21.58; SD = 4.86) (see Table 4) was at 9.78 [8.81, 10.75] 
points significantly higher (See Table 7) than the mean of pre-test score (M = 11.80, SD = 5.07) (see Table 4) in 
Data 2. 


DISCUSSION AND CONCLUSION 

The quantitative data analyses revealed that the students had significant increase in their course motivation after the 
instructions on the fuel cell content knowledge with the assistance of IGLU modules. The study results have been 
cross-validated by two data sets collected from two groups of engineering students at two separated treatment 
periods using measures in two forms, paper and online based instruments for data collections. We also found 
significant increase in student learning outcomes measured by the fuel knowledge tests; however, we want to be 
cautious to draw the causal inference at the current stage because of the absence of a control group study. We argue 
for the causality of student learning motivation changes due to the intervention which is our current study focus, but 
not for the effect on the increase of knowledge scores with the current data because participants are expected to 
acquire more content knowledge after any instructions; however, the change of learning motivations really depends 
on many course related factors. 

Students learning motivations can change positively or negatively depending on many factors related course 
instructions on students’ interest in learning. The positive effect of the IGLU intervention on student learning 
motivations revealed in the current study reflected the literature on the student interest and motivation and their 
associations with the technology use to positively impact on student learning interest and therefore to motivate them 
to learn. 


Copyright by author(s); CC-BY 


92 


The Clute Institute 














American Journal of Engineering Education - December 2016 


I 'plume 7, Number 2 


The design of the IGLU modules followed the factors defined in Keller’s ARCS Model (1987) to make a better 
learning environment to increase student situational interest in learning. Researchers of the project followed the 
ARCS model to make the content knowledge delivery (1) to attract student attention using movies, live pictures, and 
animations, (2) to be relevant to student interest by displaying the fancy technology uses to segment the process of 
fuel cell productions into interconnect subsystems, (3) build student confidence by partitioning the big concepts into 
manageable parts and lively depicting the abstract ones using animations for students to easily understand, and (4) 
satisfy digital youth’s learning needs though accommodating their learning styles with their familiar means, the use 
of instructional technologies, for content knowledge learning. 

There are many factors that affect student motivation, while the four important focuses of the IGLU modules based 
on ARCS model play a major role to improve student course learning motivations. This goal was achieved through 
managing student’s situational interest in learning by the instructor using technology. This study results are 
consistent with previous research findings in the same area. Chang (2007) stated that with the rapid development of 
technology and science, instructional technology was increasingly used over time. There are some valuable similar 
studies (e.g., Shcllnut, Knowltion, & Savage, 1999) on the positive effects using interactive modules through 
technology use in engineering instructional design. Many researchers and designers have applied games and 
animations to their curriculum and achieved positive results in terms of motivating student learning. Vogel et al. 
(2006) argued that the consistent findings about the positive effect on the cognitive gains and improvement of 
student motivations in learning were “extremely unlikely to be due to change” (p.235) based on their meta-analytic 
study of the literature. 

Based on practical experiences about our interactive modules and student motivation, we would have the same 
conclusion with Michau, Gentil, and Barrault (2001) that there is a promising view about web-based modules or the 
use of instructional technology for increasing student learning interest so that to motivate the digital generation to 
learn better. Our study results support the claims about the positive impact of popularity to use instructional 
technology, specifically for engineering students. It improved student learning motivation through addressing their 
learning habits or styles and using their favorite means to deliver content knowledge. While, we agree with some 
researchers (e.g., Walddeck, & Dougherty, 2012) that web-based or technology-based learning could be a double- 
edged sword for students’ motivation because the result is dependent on whether the technologies are used 
effectively. Admittedly, not all of the technologies will be beneficial to students’ learning motivations if not used 
appropriately. Some studies (e.g., Tai & Ting, 2015) found that some technology use may satisfy student interest, 
because students may take technology use, such as videos or games, as a way of entertainment rather than learning, 
and the leisure environment may result in an unacceptable achievement of students (Behnke & Greenan, 2011); 
therefore, the instructional technology use has to appropriately map the content knowledge with a right amount of 
usage. 

Limitations and Future Study 

There are some limitations in the current study. Even though we confidently claim the positive impact of the IGLU 
modules as an intervention on student motivation in learning, we agree that the treatment length was relatively short. 
However, with the current promising results from the relatively short period of treatment, we can reasonable assume 
more positive effect if we could have longer or more sessions to implement the interventions. Another concern could 
be that we used quasi-experimental pre and posttest designs other than the control group design. We understand that 
the quasi-experimental control group design is more rigor for us to make causal inferences from the statistical 
analysis results; however, our current study focuses on the assessment of the improvement of student learning 
motivation other than the student learning outcome. With regards to student learning motivation as the specific 
construct we studied, it is justifiable for our claims on causality of the positive effect of the program because student 
learning motivation can be both positively and negatively influenced by the instructions. 

We agree that it is beneficial for us to further assess the effects of the IGLU program on student learning. With 
regards to the statistical inference, we are planning to conduct further studies with a quasi-experiment control group 
design to assess the program effect focusing on student learning and its associations with student learning 
motivations and learning styles. The current study result echoes the exiting literature regarding learning motivation. 
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We also expect that such study will add knowledge to curriculum design through assessing the effectiveness of 
IGLU instructional modules for engineering student learning outcomes. 

With the online learning prevailing in the current university courses, the significance of the current study is that the 
education initiative of IGLU system and the assessment of its impact on student learning motivation provide 
important information for improving the modules to serve a diverse student body. It will greatly help curriculum 
development for engineering education. 


ACKNOWLEDGMENTS 

The researchers of this project thanks for the support from NSF grant #1245747. 

AUTHOR BIOGRAPHIES 

Haiyan Bai, Ph.D., is an Associate Professor in the College of Education and Human Performance (CEHP) at the 
University of Central Florida. Her research areas include issues that revolve around quantitative methods and the 
applications of statistical methods in educational research and behavioral sciences. Her publications include books, 
book chapters, articles in refereed national and international journals, and refereed international conference 
proceedings. She has also given lectures, talks, and many refereed professional presentations at the 
international/national and the regional levels and won grants from National Science Foundation’s (NSF) and U.S. 
Department of Education. 

Amjad Aman is a Ph.D. candidate at the Mechanical and Aerospace Engineering department at the University of 
Central Florida. Amjad's research has been dedicated to solid oxide fuel cells: electrochemical modeling of single 
fuel cells, structural modeling of electrolytes, material characteristic studies of the cathode material and the 
development of fuel cell educational software. 

Yunjun Xu, Ph.D. (U. of Florida, 2003) is an Associate Professor in the Department of Mechanical and Aerospace 
Engineering at the University of Central Florida. He teaches courses like Aircraft Design, Flight Mechanics, Orbital 
Mechanics, Feedback Control, and Guidance, Navigation, and Control. His research interests are robots, control 
theory, guidance, and engineering education. 

Nina Orlovskaya, Ph.D. (National Academy of Sciences of Ukraine, 1993) is an Associate Professor in the 
Department of Mechanical and Aerospace Engineering at the University of Central Florida. She teaches courses like 
Solid Mechanics, Thermodynamics, Senior Design, Science and Technology of Fuel Cells. Her research interests are 
energy conversion devices, electrochemically active ceramics, and mechanical behavior of ceramics, boron rich 
solids and engineering education. 

Mingming Zhou is a Ph.D. student in the Methodology, Measurement, and Analysis Program in the College of 
Education and Human Performance (CEDHP) at the University of Central Florida. She is a research assistant in the 
IGLU project and has also been working in the Office of Innovative Analytics and Application in CEDHP. 

REFERENCES 

Aman, A., Xu, Y., Bai, H., Orlovskaya, N., (2015, June). Interconnected software modules to aid the learning of fuel cell 
courses. Paper presented in the ASEE's 122nd Annual conference and Exposition, Seattle, WA. 

Anderson, L.W. & Krathwohl, D. R. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy 
of educational objectives. New York: Longman. 

Bai, H., Xu, Y., & Orlovskaya, N. (2014, April). Web-based interactive modules for engineering college undergraduate students. 
Paper presented at the International Conference of the Chinese American Educational Research and Development 
Association 2014, Philadelphia, PA. 

Behnke, C. & Greenan, J. P. (2011). The relationship between emotional intelligence and attitudes toward computer-based 
instruction of postsecondary hospitality students. Journal of Career and Technical Education 26(1): 62-84. 

Chang, M. (2007). Enhancing web-based language learning through self-monitoring. Journal of Computer Assisted Learning, 23, 
187-196. doi: 10.1111/j. 1365-2729.2006.00203.x 


Copyright by author(s); CC-BY 


94 


The Clute Institute 





American Journal of Engineering Education - December 2016 


I "plume 7 , Number 2 


Cordova, D. I., and Lepper, M. R. (1996). Intrinsic motivation and the process of learning: Beneficial effects of contextualization, 
personalization, and choice. Journal of Educational Psychology, 88, 715-730. 

Deci, E. L., Koestner, R., & Ryan, R. M. (1999). A meta-analytic review of experiments examining the effects of extrinsic 
rewards on intrinsic motivation. Psychological bulletin, 125(6), 627. 

Deci, E. L., & Ryan, R. M. (1975). Intrinsic motivation. John Wiley & Sons, Inc. 

Dede, C. (2004). Planning for “neomillennial” learning styles: Implications for investments in technology and faculty. Harvard 
Graduate School of Education. 

Dewey, J. (1913). Interest and Effort in Education. Riverside Press, Boston, MA 

Felder, R. M., & Silverman, L. K. (1988). Learning and teaching styles in engineering education. Engineering education, 78(1), 
674-681. 

Flowerday, T., & Schraw, G. (2000). Teacher beliefs about instructional choice. Journal of Educational Psychology, 92, 634- 
645. 

Heafner, T. (2004). Using technology to motivate students to learn social studies. Contemporary Issues in Technology and 
Teacher Education, 4( 1), 42-53. 

Hidi, S. (1990). Interest and its contribution as a mental resource for learning. Review of Educational research, 60(4), 549-571. 

Hidi, S., & Baird, W. (1986). Interestingness—A neglected variable in discourse processing. Cognitive Science, 10(2), 179-194. 

Hull, G. A., & Schultz, K. (Eds.). (2002). School's out: Bridging out-of-school literacies with classroom practice (Vol. 60). 
Teachers College Press. 

Kebritchi, M., Hirumi, A., & Bai, H. (2010). The effects of modem mathematics computer games on mathematics achievement 
and class motivation. Computers & education, 55(2), 427-443. 

Keller, J. M. (1987). Development and use of the ARCS model of motivational design. Journal of Instructional Development, 
10(3), 2 - 10 . 

Kintsch, W. (1980). Learning from text, levels of comprehension, or: Why anyone would read a story anyway. Poetics, 9(1), 87- 
98. 

Messner, J. I., & Horman, M. (2003, September). Using advanced visualization tools to improve construction education. In 
Proceedings of CONVR 2003 Conference (pp. 145-155). 

Michau, F., Gentil, S., & Barrault, M. (2001). Expected benefits of web-based learning for engineering education: examples in 
control engineering. European Journal of Engineering Education, 26, 151-168. doi: 10.1080/03043790110034410 

Mishra, P., & Koehler, M. (2006). Technological pedagogical content knowledge: A framework for teacher knowledge. The 
Teachers College Record, 108(6), 1017-1054. 

Mohler, J. L. (2001). Using interactive multimedia technologies to improve student understanding of spatially-dependent 

engineering concepts. In Proceeding of the International Conference on Computer Geometry and Graphics (Graphicon 
2001) (pp. 292-300). 

Mun, Y. Y., & Hwang, Y. (2003). Predicting the use of web-based information systems: self-efficacy, enjoyment, learning goal 
orientation, and the technology acceptance model. International journal of human-computer studies, 59(4), 431 -449. 

National Science Board (NSB). (2015). Moving forward to improve engineering education. Retrieved from: 
http://www.nsf.gov/pubs/2007/nsb07122/nsb07122.pdf. 

Nguyen, T., & Khoo, I. (2009). Learning and teaching engineering courses with visualizations. Proceedings of the 2009 World 
Congress on Engineering and Computer Science (WCECS), Vol. I, San Francisco, CA, Oct. 20-22. 

Osborne, J., Simon, S., & Collins, S. (2003). Attitudes towards science: A review of the literature and its implications. 
International Journal of Science Education, 25, 1049-1079. 

Potvin, P., & Hasni, A. (2014). Interest, motivation and attitude towards science and technology at K-12 levels: a systematic 
review of 12 years of educational research. Studies in Science Education, 50(1), 85-129. 

Raman, M., Ryan, T., & Olfman, L. (2005). Designing knowledge management systems for teaching and learning with wiki 
technology. Journal of Information Systems Education, 16(3), 311. 

Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and 
well-being. American psychologist, 55(1), 68. 

Schraw, G., & Dennison, R. S. (1994). The effect of reader purpose on interest and recall. Journal of Literacy Research, 26(1), 1- 
18. 

Schraw, G., Flowerday, T., & Lehman, S. (2001). Increasing situational interest in the classroom. Educational Psychology 
Review, 13(3), 211-224. 

Shellnut, B., Knowltion, A., & Savage, T. (1999). Applying the ARCS model to the design and development of computer-based 
modules for manufacturing engineering courses. Educational technology research and development, 47(2), 100-110. 

Shirey, L. L. (1992). Importance, interest, and selective attention. The role of interest in learning and development, 281-296. 

Subramaniam, P. R. (2010). Unlocking the power of situational interest in physical education. Journal of Physical Education, 
Recreation & Dance, 81(1), 38-49. 

Tai, Y., & Ting, Y.-L. (2015). Effects of commercial web videos on students' attitude toward learning technology. International 
Journal of Information and Communication Technology Education, 11(3), 20-29. 

Vogel, J. J., Vogel, D. S., Cannon-Bowers, J., Bowers, C. A., Muse, K., & Wright, M. (2006). Computer gaming and interactive 
simulations for learning: A meta-analysis. Journal of Educational Computing Research, 34(3), 229-243. 


Copyright by author(s); CC-BY 


95 


The Clute Institute 





American Journal of Engineering Education - December 2016 


I olume 7, Number 2 


Waldeck, J. H., & K. Dougherty (2012). Collaborative communication technologies and learning in college courses: Which are 
used, for what purposes, and to what ends? Learning, Media and Technology, 37(4), 355-378. 

Yarmolenko, S., Sankar, J., Bernier, N., Klimov, M., Kapat, J., & Orlovskaya, N. (2009). Phase stability and sintering behavior 
of 10mol% Sc203-lmol% Ce02-Zr02 ceramics. Journal of fuel cell science and technology, 6(2), 021007. 

Zimmerman, B. J., & Schunk, D. H. (Eds.) (1989). Self-regulated learning and academic achievement: Theoty, research, and 
practice. New York: Springer Verlag. 


Copyright by author(s); CC-BY 


96 


The Clute Institute 





